Data and storage communication networks are in widespread use. In many data and storage communication networks, data packet switching is employed to route data packets or frames from point to point between source and destination, and network processors are employed to handle transmission of data into and out of data switches.
FIG. 1 is a block diagram illustration of a conventional network processor in which the present invention may be applied. The network processor, which is generally indicated by reference numeral 10, may be constituted by a number of components mounted on a card or “blade”. Within a data communication network, a considerable number of blades containing network processors may be interposed between a data switch and a data network.
The network processor 10 includes data flow chips 12 and 14. The first data flow chip 12 is connected to a data switch 15 (shown in phantom) via first switch ports 16, and is connected to a data network 17 (shown in phantom) via first network ports 18. The first data flow chip 12 is positioned on the ingress side of the switch 15 and handles data frames that are inbound to the switch 15.
The second data flow chip 14 is connected to the switch 15 via second switch ports 20 and is connected to the data network 17 via second network ports 22. The second data flow chip 14 is positioned on the egress side of the switch 15 and handles data frames that are outbound from the switch 15.
As shown in FIG. 1, a first data buffer 24 is coupled to the first data flow chip 12. The first data buffer 24 stores inbound data frames pending transmission of the inbound data frames to the switch 15. A second data buffer 26 is coupled to the second data flow chip 14, and stores outbound data frames pending transmission of the outbound data frames to the data network 17.
The network processor 10 also includes a first processor chip 28 coupled to the first data flow chip 12. The first processor chip 28 supervises operation of the first data flow chip 12 and may include multiple processors. A second processor chip 30 is coupled to the second data flow chip 14, supervises operation of the second data flow chip 14 and may include multiple processors.
A control signal path 32 couples an output terminal of second data flow chip 14 to an input terminal of first data flow chip 12 (e.g., to allow transmission of data frames therebetween).
The network processor 10 further includes a first scheduler chip 34 coupled to the first data flow chip 12. The first scheduler chip 34 manages the sequence in which inbound data frames are transmitted to the switch 15 via first switch ports 16. A first memory 36 such as a fast SRAM is coupled to the first scheduler chip 34 (e.g., for storing data frame pointers (flow queues) and flow control information as described further below). The first memory 36 may be, for example, a QDR (quad data rate) SRAM.
A second scheduler chip 38 is coupled to the second data flow chip 14. The second scheduler chip 38 manages the sequence in which data frames are output from the second network ports 22 of the second data flow chip 14. Coupled to the second scheduler chip 38 are at least one and possibly two memories (e.g., fast SRAMs 40) for storing data frame pointers (flow queues) and flow control information. The memories 40 may, like the first memory 36, be QDRS. The additional memory 40 on the egress side of the network processor 10 may be needed because of a larger number of flows output through the second network ports 22 than through the first switch ports 16.
FIG. 2 schematically illustrates conventional queuing arrangements that may be provided for a data flow chip/scheduler pair (either the first data flow chip 12 and the first scheduler chip 34 or the second data flow chip 14 and the second scheduler chip 38) of the network processor 10 of FIG. 1. In the particular example illustrated in FIG. 2, the first data flow chip 12 and the first scheduler chip 34 are illustrated, but a very similar queuing arrangement may be provided in connection with the second data flow chip 14 and the second scheduler chip 38. In the queuing arrangement for the first data flow chip 12 and the first scheduler chip 34, incoming data frames (from data network 17) are buffered in the input data buffer 24 associated with the first data flow chip 12 (FIG. 1). Each data frame is associated with a data flow or “flow”. As is familiar to those who are skilled in the art, a “flow” represents a one-way connection between a source and a destination.
The first scheduler chip 34 includes flow scheduling calendars 41 (of which one is shown in FIG. 2) which define output schedules for flows which are entitled to a scheduled QoS (Quality of Service) with guaranteed bandwidth, thus enjoying high priority.
The first scheduler chip 34 also includes scheduling queues 42 (of which one is shown in FIG. 2) which arbitrates among flows entitled to a “best effort” or “available bandwidth” (lower priority) QoS. The scheduling queue 42 defines a sequence in which the flows enqueued therein are to be serviced.
As shown in FIG. 2, the flow scheduling calendar 41 and scheduling queue 42 share a respective output port 44 of the first data flow chip 12. It is to be understood that the output port 44 is one of the first switch ports 16 illustrated in FIG. 1. (However, if the data flow chip/scheduler pair under discussion were the egress side data flow chip 14 and scheduler chip 38, then the output port 44 would be one of the network ports 22.) Although only one flow scheduling calendar 41 and one scheduling queue 42 and one corresponding output port 44 are shown, it should be understood that in fact there may be plural output ports and corresponding flow scheduling calendars and scheduling queues each assigned to a respective port. (However, according to an alternative embodiment, disclosed in co-pending patent application Ser. No. 10/015,994, filed Nov. 1, 2001, a group of output ports may be associated with each scheduling queue 42. This co-pending patent application is incorporated herein by reference.)
The memory 36 associated with the first scheduler chip 34 holds flow queues made up of pointers (“frame pointers”) to locations in the first data buffer 24 corresponding to data frames associated with the flow scheduled by the flow scheduling calendar 41 or enqueued in the scheduling queue 42. The memory 36 also stores flow control information, such as information indicative of the QoS to which flows are entitled.
FIG. 3 is a schematic representation of a typical flow queue 50 stored in the memory 36. The flow queue 50 includes a header (queue identifier 52) that identifies the particular flow queue and its corresponding flow. The rest of the flow queue 50 is made up of a linked list (or “string”) 54 of frame pointers including a first frame pointer 56 (“FRAME 1”) corresponding to the head of the flow queue 50 and a last frame pointer 58 (“FRAME n”) corresponding to the tail of the flow queue 50. As indicated at 60, each frame pointer in the string 54, except for the last frame pointer 58, contains a reference to the next frame pointer in the string 54.
When the flow scheduling calendar 41 or the scheduling queue 42 indicates that a particular flow scheduled or enqueued therein is the next to be serviced, reference is made to the first frame pointer 56 of the corresponding flow queue 50, and the corresponding frame data pointed to by the first frame pointer 56 is transferred from the first data buffer 24 to an output queue 46 associated with the output port 44. The first frame pointer 56 is then removed from the flow queue 50 and the next frame pointer (“FRAME 2”) becomes the head of the flow queue 50.
When a new frame for the flow is received, the frame data is buffered in the first data buffer 24, and a corresponding frame pointer is added to the tail of the string 54 of the corresponding flow queue 50.
The memory 36 also stores a discard queue (not separately shown) which is a linked list of frame pointers corresponding to frames stored in the first data buffer 24 that are to be discarded without being transmitted. As is familiar to those who are skilled in the art, frames stored in the first data buffer 24 may be discarded because they are in excess of the number of frames allowed to a flow according to its Quality of Service. Alternatively, frames may be discarded when no flow queue is available to service the frames or when the network processor 10 simply has to dispose of a certain number of frames due to network congestion or the like.
Typically the discard queue is serviced on a scheduled basis, with sufficient priority that the discard queue is kept close to an empty condition, so as to promptly free up the space occupied by the frames to be discarded in the first data buffer 24.
As is also familiar to those who are skilled in the art, it becomes necessary from time to time to disable a flow, as when the connection represented by the flow is terminated, and then to reconfigure the flow so as to serve a new connection. In accordance with conventional practices, a disable bit is written to the flow control structure (stored in the memory 36) corresponding to the flow to be disabled. This prevents any new frames from being enqueued to the flow. Such new frames are, instead, enqueued to the discard queue. Frames already enqueued to the flow are serviced in ordinary course according to the QoS to which the flow is entitled. A queue byte count, which indicates the amount of data in the flow (buffered in the first data buffer 24) is monitored. As the frames enqueued to the flow are transmitted, the associated buffer space in the first data buffer 24 is released for use, and the queue byte count decreases. When the queue byte count reaches zero, indicating that the flow is empty, the flow may then be reconfigured for use by another connection. As is appreciated by those who are skilled in the art, the reconfiguring of the flow may involve changes in the QoS, the output port, and other flow parameters.
A disadvantage of the conventional procedure for disabling a flow is that reconfiguration of the flow must await transmission of all enqueued frames to empty the flow queue. This delay may adversely affect performance of the network processor. Furthermore, there is overhead involved in repeatedly reading the queue byte count until the flow has emptied and the queue byte count has reached zero. It would be desirable to reduce the amount of time required to reconfigure a flow after the flow is disabled.